1. Field
Embodiments of the invention relate to the field of flow monitoring; and more specifically, to a more intelligent and dynamic sampling of network flows.
2. Background
Network flows are monitored for a variety of purposes, including usage-based accounting, traffic profiling, traffic engineering, attack or intrusion detection, and QoS (quality of service) monitoring. One method of monitoring flows is performed by reporting on each and every flow. While this implementation provides an accurate and complete picture of network traffic, it requires a large number of resources, including memory and CPU (central processing unit) usage on the forwarding plane of the network device, backplane bandwidth to send flow records to the control plane of the network device and also out to an external collector, memory and CPU usage on the control plane of the network device, network bandwidth consumption due to heavy communication with the external collector, and memory and CPU usage on the external collector. Reporting on each and every flow is not feasible in high-speed environments.
An alternative to complete reporting of all flows is to report on a sample of the flows. For example, sampling can be done in packet intervals. Packet interval sampling takes a random selection of a subset of packets, e.g., by reporting on 1 out of every 1,000 packets received on a network device. However, there are various drawbacks of packet interval sampling. One drawback is that this sampling does not provide a precise number of packets in a flow. If the number of packets reported is 1,000, one can only assume that there were approximately 1,000,000 packets in the flow, but it is not an exact count. Another drawback is that packet interval sampling tends to report longer flows rather than shorter flows because if a flow contains fewer than 1,000 packets, a sample may not necessarily be taken from this flow. For these reasons, packet interval sampling is not ideal.
An alternative to packet interval sampling is flow interval sampling. Flow interval sampling takes a random selection of a subset of flows, e.g., by reporting 1 out of every 5 flows, and reporting the selected flows in their entirety. This sampling resolves the aforementioned drawbacks of packet interval sampling. However, with both packet interval sampling and flow interval sampling, the reporting is not as complete as monitoring and reporting on each and every flow.